Optimizing the integration of agent-based cloud archestrators and higher-level workloads

Part 5: Ph.D. Track: Autonomic and Self-Management SolutionsInternational audienceThe flexibility of cloud computing has put significant strain on operations teams. Manually installing and configuring applications in the cloud simply isn’t an option anymore. Configuration management automation solves the issue of getting a single application into a certain state automatically and reliably. However, the issue of automatic dependency management between multiple applications is still an “open, hard problem” according to researchers at Google. Agent-based modeling and orchestration tools like Juju solve the issue of getting from zero to a working set of correctly clustered and connected frameworks. The shortcomings of these state-of-the-art tools are that they don’t provide efficient ways to model and orchestrate workloads running on top of these frameworks. This paper presents a number of ways to deploy and orchestrate workloads with Juju, compares their performance and overhead, and suggests how this overhead can be minimized

Distributed service orchestration : eventually consistent cloud operation and integration

Both researchers and industry players are facing the same obstacles when entering the big data field. Deploying and testing distributed data technologies requires a big up-front investment of both time and knowledge. Existing cloud automation solutions are not well suited for managing complex distributed data solutions. This paper proposes a distributed service orchestration architecture to better handle the complex orchestration logic needed in these cases. A novel service-engine based approach is proposed to cope with the versatility of the individual components. A hybrid integration approach bridges the gap between cloud modeling languages, automation artifacts, image-based schedulers and PaaS solutions. This approach is integrated in the distributed data experimentation platform Tengu, making it more flexible and robust

Beyond generic lifecycles : reusable modeling of custom-fit management workflows for cloud applications

Automated management and orchestration of cloud applications have become increasingly important, partly due to the large skills shortage in IT operations and the increasing complexity of cloud applications. Cloud modeling languages play an important role in this, both for describing the structure of a cloud application and specifying the management actions around it. The TOSCA cloud model standard recently defined declarative workflows as the preferred way to specify these management actions but, as noted in the standard itself, this is far from ideal. This paper draws lessons from six years of using declarative workflows in Juju for deploying and managing complex platforms such as OpenStack and Kubernetes in production. This confirms the limitations: declarative workflows are inflexible, hard to reuse, and allow for related components to become silently incompatible. This paper proposes the reactive pattern to solve these issues by enabling the creation of emergent workflows using declarative flags and handlers, which can be easily grouped into reusable layers. After more than two years of using this pattern in production as part of our charms. reactive framework, it is clear that it enables reusability and ensures compatibility: 67% of reactive charms share parts of the management workflow and 73% of reactive charms share a relationship workflow

FUSE : a microservice approach to cross-domain federation using docker containers

In crisis situations, it is important to be able to quickly gather information from various sources to form a complete and accurate picture of the situation. However, the different policies of participating companies often make it difficult to connect their information sources quickly, or to allow software to be deployed on their networks in a uniform way. The difficulty in deploying software is exacerbated by the fact that companies often use different software platforms in their existing networks. In this paper, Flexible federated Unified Service Environment (FUSE) is presented as a solution for joining multiple domains into a microservice based ad hoc federation, and for deploying and managing container-based software on the devices of a federation. The resource requirements for setting up a FUSE federation are examined, and a video streaming application is deployed to demonstrate the performance of software deployed on an example federation. The results show that FUSE can be deployed in 10 minutes or less, and that it can support multiple video streams under normal network conditions, making it a viable solution for the problem of quick and easy cross-domain federation

Design and evaluation of a scalable Internet of Things backend for smart ports

Internet of Things (IoT) technologies, when adequately integrated, cater for logistics optimisation and operations' environmental impact monitoring, both key aspects for today's EU ports management. This article presents Obelisk, a scalable and multi-tenant cloud-based IoT integration platform used in the EU H2020 PortForward project. The landscape of IoT protocols being particularly fragmented, the first role of Obelisk is to provide uniform access to data originating from a myriad of devices and protocols. Interoperability is achieved through adapters that provide flexibility and evolvability in protocol and format mapping. Additionally, due to ports operating in a hub model with various interacting actors, a second role of Obelisk is to secure access to data. This is achieved through encryption and isolation for data transport and processing, respectively, while user access control is ensured through authentication and authorisation standards. Finally, as ports IoTisation will further evolve, a third need for Obelisk is to scale with the data volumes it must ingest and process. Platform scalability is achieved by means of a reactive micro-services based design. Those three essential characteristics are detailed in this article with a specific focus on how to achieve IoT data platform scalability. By means of an air quality monitoring use-case deployed in the city of Antwerp, the scalability of the platform is evaluated. The evaluation shows that the proposed reactive micro-service based design allows for horizontal scaling of the platform as well as for logarithmic time complexity of its service time

City of things : enabling resource provisioning in smart cities

In the last few years, traffic over wireless networks has been increasing exponentially due to the impact of IoT. IoT is transforming a wide range of services in different domains of urban life, such as environmental monitoring, home automation, and public transportation. The so-called smart city applications will introduce a set of stringent requirements, such as low latency and high mobility, since services must be allocated and instantiated on demand, simultaneously, close to multiple devices at different locations. Efficient resource provisioning functionalities are needed to address these demanding constraints introduced by smart city applications while minimizing resource costs and maximizing QoS. In this article, the CoT framework is presented, which provides not only data collection and analysis functionalities but also automated resource provisioning mechanisms for future smart city applications. CoT is deployed as a smart city test-bed in Antwerp, Belgium, which allows researchers and developers to easily set up and validate IoT experiments. A smart city use case of air quality monitoring through the deployment of air quality sensors in moving cars is presented showing the full applicability of the CoT framework for a flexible and scalable resource provisioning in the smart city ecosystem

Initiating a collaborative monitoring system to survey Maltese orchids

Orchids have always garnered the interest of enthusiastic amateurs and nonprofessional researchers as a charismatic species group and have become a flagship for Mediterranean flora. At the same time they are a decent indicator for biodiversity and environmental quality through their association with specific pollinators and mycorrhizal fungi. Historical records for orchids occurring in the Maltese Islands number some 30 different species, but there is little to no comprehensive data on their abundance and distribution.peer-reviewe

Collaborative compositions : facilitating service orchestration from cloud to edge

Samenwerking overheen organisatorische grenzen wordt steeds belangrijker. Dit gaat zowel over samenwerking tussen teams onderling, als tussen organisaties en bedrijven. De vierde industriële revolutie speelt hier een grote rol in. Deze start namelijk door digitale systemen die de fysieke wereld manipuleren te verbinden. De complexiteit van IT systemen stijgt enorm door de vergaande digitalisering van de maatschappij. Om deze complexiteit de baas te kunnen, wordt software steeds meer ontwikkeld als collaboratieve composities: verschillende componenten die samenwerken om één dienst te leveren. Applicaties zoals Netflix of Google bestaan uit honderden tot duizenden componenten die samenwerken om hun functionaliteit te voorzien. Dit proefschrift onderzoekt hoe software systeembeheerders kan helpen om collaboratieve composities te installeren en te beheren. Het stelt technieken en software voor die systeembeheerders de mogelijkheid geven om hun kennis in software te encapsuleren zodat ze deze kunnen delen en hergebruiken. Ook onderzoekt dit werk hoe huidige ontwikkelingen in het beheren van cloud applicaties kunnen toegepast worden op het beheer van applicaties die dichter bij gebruikers draaien, in de edge en de fog